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Abstract 

One key requirement for fountain (rateless) coding schemes is to achieve a high intermediate 
symbol recovery rate. Recent coding schemes have incorporated the use of a feedback channel to 
improve intermediate performance of traditional rateless codes; however, these codes with feedback 
are designed based on uniformly at random selection of input symbols. In this paper, on the other 
hand, we develop feedback-based fountain codes with dynamically-adjusted nonuniform symbol selection 
distributions, and show that this characteristic can enhance the intermediate decoding rate. We provide 
an analysis of our codes, including bounds on computational complexity and failure probability for 
a maximum likelihood decoder; the latter are tighter than bounds known for classical rateless codes. 
Through numerical simulations, we also show that feedback information paired with a nonuniform 
selection distribution can highly improve the symbol recovery rate, and that the amount of feedback 
sent can be tuned to the specific transmission properties of a given feedback channel. 

Index Terms 

Fountain codes. Feedback channel, LT codes. Nonuniform symbol selection 

I. Introduction 

Reliable communication over erasure channels has emerged as a key technology for various networked 
applications, for example digital video broadcasting and over-the-air software updates. In applications 
where there exists a high-throughput feedback channel, automatic repeat request (ARQ) protocols guar¬ 
antee reliability over erasure channels. However, when such feedback channels are not available, rateless 
codes, such as the capacity achieving Luby-Transform (LT) [3] and Raptor codes [4], can often provide 
reliable communication for sufficiently long block lengths. These codes have a well-known all-or-nothing 
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Fig. 1. Two-step rateless encoder with a degree distribution and nonuniform symbol selection distribution. 


decoding property (the so-called “waterfall” phenomenon), where a jump in the fraction of decoded input 
symbols occurs near the very end of the decoding process. For applications with real-time requirements, 
however, it is desirable to be able to recover symbols as decoding proceeds, i.e., to achieve a high 
intermediate symbol recovery rate. 

In fact, the intermediate performance of classical codes can be improved by incorporating the use of a 
feedback channel. For instance, a decoder in Real-Time (RT) oblivious [5] and Shifted-LT (SLT) [6] codes 
sends the number of recovered symbols back to the transmitter, and this feedback is used to modify the 
degree distribution at the encoder. Previous feedback-based rateless codes are mostly based on adjusting 
the degree of encoding symbols, e.g., by shifting the degree distribution in the SLT codes. However, 
after a degree d is picked for an encoding symbol, d input symbols are chosen uniformly at random and 
xored to form the symbol. Moreover, the encoder does not have full freedom in controlling the number 
of feedbacks transmitted. 

In this paper, we develop a class of rateless coding schemes that optimize for high intermediate symbol 
recovery rate. At its core, our encoder uses a nonuniform selection distribution that is dynamically adjusted 
based on feedback information. Fig. 1 depicts a schematic of our two-step encoder, where we illustrate 
that the inputs are chosen according to a feedback-based selection distribution, rather than uniformly at 
random. Feedback messages contain information on the distance between a received encoding symbol 
and the set of already decoded symbols at the receiver. In the general form of our codes, the encoder 
estimates the probability that each input symbol has been decoded (at the decoder), and these estimates 
are then used to dynamically tune the selection of input symbols within subsequent transmissions. This 
method enables the encoder to naturally track the decoding progress and generate encoding symbols that 
result in a faster decoding rate compared with a uniform selection of input symbols. This class of codes 
is suitable for the scenarios with relatively large feedback budgets, although we allow the decoder to 
specific control when feedback occurs (according to the budget). 

On the other hand, the primitive form of our code is designed based on a parsimonious use of the 
feedback channel. In this case, the encoder learns which symbols have been decoded, and those symbols 









will be assigned with a selection probability of zero for subsequent encodings. This coding scheme is 
suitable for applications with limited feedback capacity such as satellite networks [7], as we require the 
decoder to opportunistically send just one bit of feedback when certain conditions are met. Note that the 
coding schemes proposed in this work are presented as enhancements of LT codes for the case some 
feedback communication is available. The motive to base our codes on the LT degree distributions is to 
accommodate cases where feedback is extremely limited or completely unavailable, in which case we 
fall back to the standard LT performance. That said, the same methodology can apply to different rateless 
codes in the literature, and to others to be proposed in the future. 

A. Organization 

The rest of this paper is organized as follows. In Section II we describe the problem setup and 
review various related coding schemes. Section III presents the most general form of our coding scheme. 
Section IV describes the primitive form of our codes adapted for constrained feedback applications. 
Coding analysis for short block lengths is presented in Section V, followed by maximum likelihood 
decoder analysis in Section VI. Simulation results are presented in Section VII. We conclude with overall 
thoughts in Section VTTT. 


IT Background 

This section describes the problem setup and some previous work on rateless coding. 

A. Preliminaries 

In the rateless coding setup, it is assumed that an encoder (broadcaster) has k input symbols to transmit 
to all receivers over an erasure channel, and that there may exist a feedback channel through which 
receivers can send some information back to the encoder. Luby Transform (LT) rateless codes [3], as the 
first practical realization of fountain codes, support full recovery of k input symbols using an expected 
number of k + O (^y/kln^{k/5)^ error-free transmissions with a given recovery failure probability 5. To 
generate an output symbol, the encoder first picks a coding degree d according to the Robust Soliton 
distribution [3]. Next, d input symbols are chosen uniformly at random without replacement, and their sum 
over an appropriate finite field forms fhe oufpuf symbol. Indices of fhe d selecfed inpuf symbols, referred 
fo as neighbors of fhe oufpuf symbol, are made available (i.e., as mela-informalion) fo fhe decoder. In 
fofal, fhe coding operafions incur fhe computafional cosf of O {k\n{k/5)). 

The LT decoder (so-called Peeling decoder) uses a simple message passing algorifhm, wifh a complexify 
fypically less fhan fradifional Gaussian elimination methods. In one variant, the decoder finds all encoding 



symbols with degree 1, whose neighbor can be immediately recovered. These recovered input symbols 
are then excluded from all output symbols that have them as neighbors, reducing the number of unknowns 
in those encoding symbols by one. This process continues until there exists no encoding symbol with 
degree 1. Decoding succeeds if all input symbols are recovered; alternatively, decoding fails if, at some 
point, there is no output symbol with degree 1. 

B. Related work 

Both fixed rate low-density parity-check (LDPC) codes [8] and Turbo codes [9] are capable of correcting 
bit errors, as well as erasures. Byers et al. in [10] have presented fixed rate Tornado codes as a class of 
simplified capacity-achieving LDPC codes. Within the context of rateless coding, random linear codes 
(see, for example, [11]) are well known due to their low communication overhead, but the encoding 
and decoding computations make them practical only for small message sizes. On the other hand, Luby 
Transform (LT) [3] codes and their extensions such as Raptor codes [4] are examples of rateless codes that 
are asymptotically optimal and also have computationally efficient encoding and decoding algorithms; 
unfortunately, they usually have poor performance for small block sizes [12] and various optimization 
methods (e.g., [13]) have been proposed for these cases. 

In some applications, like video streaming, intermediate symbol recovery is important, as it is desirable 
to decode some symbols before an entire frame has been received. The authors in [14] design a degree 
distribution for high intermediate symbol recovery rates. Recently, there have also been proposed rateless 
protocols that utilize side information fed back from the decoder to the encoder. Based on the type of 
feedback used, they can be divided in the following categories: 

• the receiver sends the number of decoded symbols to the transmitter; 

• the receiver suggests to the transmitter what kind of degrees it should use for future encodings; or 

• the receiver notifies the transmitter of which input symbols have been recovered. 

In the Real-Time (RT) oblivious codes [5], the encoder starts with degree one symbols, and it increments 
the degree of encoding symbols based on feedback messages. In this case, feedbacks contain information 
on the number of recovered symbols. Shifted LT (SLT) codes proposed in [6] use the same type of 
feedback information as the RT codes, but instead of explicitly increasing the encoded symbols degree, 
the encoder shifts the Robust Soliton degree distribution. There also exist rateless-type codes with real¬ 
time properties that allow intermediate knowledge of some input symbols as the decoding progresses. 
The authors in [15] propose Growth codes for the data collection within lossy sensor networks. Similar 
to the RT and SLT codes. Growth codes’ degree increases as the coding progresses. 



As another type of feedback, the receiver in [16] has the ability to control the decoding progress 
by requesting particular degrees. In this method, the average number of output symbols required for 
decoding k input symbols is shown to be upper bounded by 1.236A:. Yet another type of feedback in 
[12] contains the identity of recovered symbols, which are used by the encoder to redesign the degree 
distribution for subsequent transmissions. Recently, the authors in [17] have proposed a heuristic to use 
a hybrid feedback-based rateless codes, called LT-AF, in which the receiver alternates between two types 
of feedback messages: the first type of feedback contains the number of decoded symbols as in the SLT 
and RT codes, while the receiver requests a specific inpuf symbol fhrough fhe second type of feedback. 

In fhis paper, fhe fype of feedback used is based on disfance informalion by which fhe encoder learns 
abouf fhe stale of individual symbols at the decoder side. Based on the feedback information, the encoder 
tunes a nonuniform selection distribution to choose neighbors of encoding symbols. This is a key point 
as all previous rateless codes are built upon a uniformly at random selection of neighbors. Moreover, we 
do not assume that the feedback channel is high bandwidth; instead, we strive for a parsimonious use of 
the feedback channel. Indeed, in the primitive form of our codes, the feedback is exactly one bit (plus 
some header information) for each of a small fraction of received symbols. 

111. Nonuniform Rateless Codes: General Form 

Previous rateless codes with feedback are mostly designed based on modifying the output symbols 
degree distribution according to feedback information, e.g., by shifting the degree distribution, or by 
explicitly increasing the degree. In these schemes, when a coding degree d is picked, d input symbols 
are selected uniformly at random to construct an encoding symbol. Moreover, in most of previous works, 
feedback information does not provide a complete picture of the decoding state at the receiver side. 
For instance, sending the number of recovered symbols in [5,6] does not provide information about 
the decoded symbols themselves. Within this context, we present a nonuniform rateless coding scheme 
wherein various input symbols are selected based upon a nonuniform distribution. In particular, the 
selection distribution is tuned according to feedback messages, which contain the distance of received 
symbols to the set of already recovered symbols at the receiver. The definition of distance quantity is as 
follows: 

Definition. Given a set of recovered symbols C and an encoding symbol y that has a set of neighbors 
A, the distance between y and C is defined as: 

dist{y,C) = ^ 

Xi£A 

where is an indicator function that is equal to 1 if and only if x is true. 
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Fig. 2. Distance graph labeling: A label {xi,qi) implies that the input symbol Xi has been decoded with probability qi up to 
the current state. Labels of output nodes yi are defined to be the number of neighbors of yi with a label of less than 1. 

The distance quantity simply corresponds to the number of neighbors of y that are not already decoded. 
As an example, suppose that input symbols xi, X 4 are encoded and transmitted in the following order: 
yi = xi + X2, y2 = xi + X4, ys = X4, and y^ = xi + X2^- The distance from either yi or y2 to the set 
of recovered symbols is 2 ; thereafter, from y^ the distance is 1 , and finally, from 2/4 the distance is 0 (as 
xi and X 2 will be decoded after receiving 1 / 3 ). 

Ultimately, the goal of the encoder is to generate encoding symbols based on the state of the decoder 
in such a way that more “helpful” symbols have a higher selection probability. To this end, the encoder 
uses distance information to estimate the probability that each input symbol has been decoded (at the 
receiver), and these estimates are used to bias the selection of input symbols. In this approach, the receiver 
can adjust the number of feedbacks using a parameter s so that one feedback transmission follows after 
every s received encoding symbols. The parameter s can be set to any arbitrary value, depending on the 
feedback channel available. 

A. Processing distance information 

In order to process distance information, the encoder constructs a bipartite graph wherein input symbols 
are placed on the top and encoding symbols at the bottom, as shown in Fig. 2. In this graph, labels are 
assigned to input and output symbols. In particular, label of an input symbol corresponds to its probability 
of having been decoded, while label of an output symbol y represents the number of neighbors of y with 
label less than 1. For instance, assume that after t feedbacks, rit neighbors of y are labeled 1 (i.e, they 
have been decoded). Therefore, the label of y, denoted by It, is calculated as 

lt = d-nt, (1) 

where d is the degree of y. In this equation, the encoder excludes the recovered neighbors from the 
labeling process. Next, in order to calculate the label of an input symbol, we assume that the f-th 

*For the sake of clarity, we assume that encoding and decoding are performed over the field F 2 . 




Fig. 3. The encoder estimates the probability of having been decoded (i.e., qj’s) for input symbols. Probability 1 (white color) 
implies that the symbol has been decoded, while probability 0 (black color) shows that the symbol has not been decoded yet. 


feedback message contains the distance ft corresponding to the encoding symbol y = 
label of a constituent symbol Xj is then defined as: 


f ('//)! f It-ft] 

= max j = max | • (2) 

It should be noted that It is the number of neighbors with a label less than 1 and ft is the number 
of undecoded neighbors. Therefore, the probability of having the neighbor j decoded is calculated as 

7 _ r 

* . Finally, after receiving a new feedback message, qj is updated to the maximum of its previous 

value and the calculated probability at the current step. For instance, assume that the encoding symbol 
y = xi + X 2 + T 3 + X 4 has a distance of 2 with the current state of decoder, meaning that two neighbors of 
y have not been decoded yet (the encoder does not know which two symbols). If the encoder has already 
assigned label 1 to xi (i.e., xi has been decoded), then the encoder uniformly divides the distance of 2 
between the remaining symbols (i.e., X 2 , x^, and xf), suggesting that each of them has been decoded 
with probability 2^ = 2. It should be noted that the subscript t in qj^t represents the evolution of q as 
the coding proceeds. For simplicity, we drop it in our discussion. 

Our labeling process tracks the state of the decoder by answering this question: what is the probability 
that an individual symbol Xj has been decoded up to this point? As an example. Fig. 3 shows a realization 
of input symbols at the encoder, where input symbols are assigned with a probability of having been 
decoded. In this case, qj = 1 (white color) implies that symbol j has been recovered, while qj = 0 (black 
color) means that symbol j has not been recovered yet. Therefore, input symbols are assigned with a 
weight between 0 and 1, which is used in the selection distributions defined in Section III-B. 

To examine the accuracy of the estimated probability values against the actual decoder state, we use 
the Mean Absolute Error (MAE) quantity. Assume that hj is an indicator function representing the state 
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Fig. 4. Mean Absolute Error (MAE) for the estimated probability of having been decoded as the interval of feedback transmission 


(parameter s) increases. 

of symbol j at the decoder such that bj = 1 if symbol j has been decoded and bj = 0 otherwise. MAE 
is then calculated as: 



in which qj’s are estimated using (2). Fig. 4 shows the MAE quantity averaged over all feedback messages 
transmitted as the interval of feedback transmission (i.e., parameter s) increases. The results illustrate 
that decreasing the interval of feedback transmission (i.e., higher feedback rate) decreases the estimation 
error. 

Remark 1 (cumulative feedback information): Distance messages accumulate information across all 
received feedbacks. Specifically, assume that there exist k input symbols at the encoder, and after 
receiving a new feedback message, say the f-th feedback, the encoder updates the probability vector 
qt = {qi,t,<l2,t, where qj^t is the probability that the input symbol j has been decoded. The 

encoder updates probability values corresponding to neighbors of the encoding symbol, and other prob¬ 
ability values remain unchanged. This update mechanism allows the encoder to accumulate information 
across all feedback messages, noting that in previous feedback-based schemes (e.g., sending number of 
recovered symbols in [5,6]), a new feedback makes previously received feedback information obsolete. 

Distance feedbacks provide implicit information about the decoder’s graph; however, it should be noted 
that one can envision other techniques to learn about the decoding graph. For instance, the decoder can 
send the whole decoding graph back to the encoder, and thus, the encoder would have full knowledge 
about the state of input symbols at the decoder’s side. This method, however, incurs high communication 










overhead on the hack channel. Next, we define nonuniform symbol selection distributions based on 
probability values g/s. 

B. Nonuniform symbol selection 

We discussed that the encoder uses distance information to learn about the state of decoder. For the 
sake of concreteness, assume that the encoder estimates the input symbol xj has been decoded with 
probability qj, and it has probability pj to be included in the next encoding symbol with degree d. 
We aim to design a symbol selection distribution that picks those d input symbols that can achieve a 
maximum decoding progress. Specifically, the selection distribution should select d — 1 symbols which 
have been recovered with a high probability, and a single symbol that has not been recovered with a high 
probability. To put in a formal framework, we have the following definition. 

Definition. For a given input symbol x and a set of d — 1 input symbols A, the Decoding Probability 
function DP{x, A) is defined as the probability of immediate decoding the input symbol x after receiving 
y = x + EiGA xi- 

In order to decode an input symbol xj within a transmission, the transmitted symbol with the degree 
of d should include the undecoded symbol Xj and d — 1 already decoded symbols. Symbol Xj is not 
decoded with probability 1 — qj, and d — I symbols belonging to the set A have already been decoded 
with probability HiG/l Therefore, at each step to transmit a symbol of degree d, the encoder should 
choose d input symbols (^Xj,A*^ satisfying: 

(x*, A*) = argmax DP(xj,A) = argmax(l — qf qi. (3) 

(xjA) {xj,A) i(zA 

The solution of this optimization problem is deterministic; in other words, the encoder always picks d—1 
symbols with the largest value of q xored with a single symbol with the smallest value of q. However, 
it is desirable to preserve the same behavior with a probabilistic scheme such that if an input symbol j 
is included in the solution of the deterministic formulation, it would also have a high probability to be 
picked by the probabilistic method. This results in the following scheme to define the selection probability 
Pf 

j I- qj if 0 ^ qj < i; 

Pj ot S 

I qj otherwise. 

In the second step of designing the selection distribution, we note that a single unrecovered (with high 
probability) symbol should be included within each encoding symbol. Therefore, based on the value of 
qj’s, input symbols are divided into two subsets: U containing undecoded symbols, and D containing 



decoded symbols. Input symbols with 0 ^ g < ^ are included in U and the rest are added to the set 
D, and thus we may construct an encoding symbol of degree d by selecting one symbol from U based 
on the selection distribution Pjj, and d — 1 symbols from D according to the distribution Pd. Selection 
distributions Pjj and Pd are defined as follows: 


PuU) = { 




if j G U; 

otherwise. 


Qj 


Pd{j)={ ^ 
0 


if j G D; 
otherwise. 


( 4 ) 


In the distributions, m is the size of subset D. Finally, the encoder transmits the xor of d selected symbols. 

This scheme based on the distributions Pd and Pu is refereed to as the All-Distance codes since all 
distance feedbacks are needed to estimate probability value g^’s. Next, we relax this scheme in a way 
that, instead of sending all distance values, the decoder quantizes distance values and allocates only a 
single bit feedback for each received encoding symbol. 


C. Quantized distance codes 

The All-Distance codes work based on estimating probability values g/s from distance information. 
In this case, at most nlog(dmax) bits are sent back to the encoder, as each of n encoding symbols can 
have distance dmax, which is the maximum degree of an encoding symbol, noting that dmax can be at 
most equal to k. 

To limit amount of feedback, we consider a scheme with a single bit feedback per received encoding 
symbol. In particular, this scheme is based on the same idea of splitting input symbols into two subsets; 
however, instead of having an exact estimation of probability value qj’s, the decoder decides to send a 
feedback 0 or 1 based on the distance of a received symbol. More precisely, the decoder calculates the 
ratio of distance to degree for a received symbol, and if the ratio is larger than 4, it implies that majority 
of neighbors within the received encoding symbol have not been recovered. In this case, the decoder 
allocates a single bit of 0 as the feedback message. On the other hand, if the calculated ratio is smaller 
than 4, it shows that majority of neighbors have been decoded and feedback message would be 1. To 
limit the number of feedback transmissions, the receiver bundles the 1-bit feedback messages together 
for every interval of s received encoding symbols, and sends the s-hit messages back to the encoder. 

At the encoder side and upon receiving a feedback message 0, corresponding neighbors are assigned 
with Qj = 0 and thus added to the subset U. Conversely, if the received feedback contains a bit of 1, 
corresponding neighbors are assigned with qj = 1 and grouped into D. This quantized version of qj is 
equivalent to evaluating in (4) ([x] rounds x to its nearest integer). As a result, the Pu and Pd 
distributions would become uniformly distributed over the subsets D and U respectively. However, it 



should be noted that with a high probability only a single undecoded symbol is included within each 
transmission. Hence, splitting a single uniform distribution defined over all input symbols (as it has been 
used in previous rateless codes) into two disjoint uniform selection distributions, can significantly improve 
the intermediate performance of rateless codes. 

In terms of total amount of feedback, decoder sends exactly one bit feedback per received encoding 
symbol, where the total number of encoding symbols is (1 + e)A: for a small value of e. Recall that 
the motivation behind the distance type feedback is to learn about the state of individual symbols at 
the decoder side. However, an alternative and trivial solution includes sending the identity of recovered 
symbols back to the encoder with potentially more feedback that could be up to A:log(A:) bits. In this 
case, it may not be clear how the encoder uses deterministic information on the identity of recovered 
symbols. In fact, the authors in [12] use the identity of recovered symbols in order to redesign the primary 
degree distribution through a computationally expensive algorithm; on the other hand, we use distance 
information through a probabilistic scheme to dynamically assign nonuniform selection weights to input 
symbols. 


IV. Nonuniform Rateless Codes: Primitive Form 

In most of communication systems, a nominal utilization of the back channel is desirable as the 
bandwidth is mainly provisioned for forward transmissions. In the previous section, we presented a 
nonuniform coding scheme based on distance feedbacks, wherein all distance information are fed back 
to the encoder. In the scheme based on quantized distance information, decoder needs one bit feedback 
per received encoding symbol. In this section, we establish a coding scheme called Delete-and-Conquer 
with a more limited use of the feedback channel. In this case, the decoder is allowed to transmit one bit 
feedback for a small fraction of received encoding symbols, when certain conditions are met. 

A. Delete-and-Conquer codes 

Recalling the definition of the distance metric, a distance 0 happens if and only if all neighbors of 
the received encoding symbol have already been decoded. Similarly, a distance 1 occurs in the case that 
there is only a single undecoded neighbor, which can then be recovered uniquely. In other words, a 
distance of 0 or I provides information about the recovery of neighbors that are part of a received linear 
combination. 

A Delete-and-Conquer encoder performs similar to the LT encoder in that it first picks a coding degree 
d from the degree distribution. However, in the second step the encoder selects d symbols from a subset of 
input symbols. Specifically, upon receiving a feedback message, the encoder assigns a selection probability 



Algorithm 1 Delete-and-Conquer Encoding (xi, X 2 ,Xfc) 
1: z 0 and m ■(— 0 

2: A <r- {xi, X 2 , Xk} and B $ 

3: while z < A: do 

4: Pick a coding degree d from the distribution Qk-m 

5: Select d symbols uniformly at random from set A 

6: Send symbol y as XOR of d selected symbols 

7: if feedback(y) = true then 

8: C Neighbors of y 

9: B ^ BUC 

10 : m 1^1 

11 : A^A\B 

12: end if 

13: if Terminate = true then 

14: Z = k 

15: end if 

16: end while 


of zero to the neighbors of the acknowledged encoding symbol, while remaining symbols would have 
an equal selection probability. Intuitively, the encoder deletes recovered symbols and continues with a 
smaller block of symbols; in so doing, the encoder also rescales the primary degree distribution (e.g., 
the Robust Soliton disttibution denoted by to the smaller set of input symbols with size k — m, 
in which m is the number of deleted symbols. Excluding recovered symbols from future transmissions 
reduces the computational complexity at the encoder and decoder. Algorithm 1 gives the pseudo-code of 
the Delete-and-Conquer encoding scheme. 

The Delete-and-Conquer decoder is based on Peeling decoder with a slight modification that upon 
receiving a new encoding symbol, the decoder checks if the distance is equal to 0 or 1. The 0 and 1 
distance feedbacks are indeed a generalization of the traditional acknowledgment to the coded cases in 
that they notify the recovery of a group of input symbols involved in an encoding. The pseudo-code of 
the Delete-and-Conquer decoding is provided in Algorithm 2. 

Remark 2 (probabilistic feedback): In the case of severe constrained feedback, the receiver adds the 
mechanism of probabilistic feedback control, in which feedbacks are only transmitted with a given 
probability. An optimal feedback probability can be determined according to the capacity of back channel 





Algorithm 2 Delete-and-Conquer Decoding of k symbols 
1: 5 0 > 5 is the set of recovered symbols 

2: while |5| < /c do 

3: y Received encoded symbol 

4: if Distance(y, 5) = 0 or 1 then 

5: Send a feedback and set feedback(y) true 

6 : end if 

7: call Peeling-Decoder 

8: Update S 

9: if |5| = k then 

10: Terminate = true 

11: end if 

12: end while 
13: 

14: function Distance(?/, S) 

15: distance 0 

16: for all neighbors Xi of y do 

17: if Xi ^ S then 

18: Increment distance 

19: end if 

20 : end for 

21: return distance 

22 : end function 


and the cost of feedback transmission. For instance, Fig. 5(a) shows simulation results of the coding 
overhead (i.e., number of forward transmissions normalized with respect to the number of input symbols) 
as the probability of sending 0 and 1 feedbacks increases. The results illustrate that when the probability 
of sending 0 and 1 distance feedbacks increases, amount of forward communications decreases. On the 
other hand, as Fig. 5(b) shows, the (normalized) number of transmitted feedback messages increases with 
the probability, as expected. Therefore, by adjusting the probability of feedback transmission, decoder 
would be able to control the number of forward and feedback transmissions. 

Note that the Delete-and-Conquer encoder learns about the recovered symbols using a light-weight 
feedback and excludes the recovered symbols from subsequent transmissions. Alternatively, the receiver 







(a) Number of forward transmissions (b) Number of feedbacks 

Fig. 5. (a) Number of forward transmissions (normalized to the number of input symbols) needed by Delete-and-Conquer codes 
as the probability of sending feedback increases (b) Number of feedback (normalized to the number of input symbols) as the 
probability of sending feedback increases. 


can send the identity of recovered symbols back to the transmitter. In this case, however, total amount 
of feedback (up to A:log(A;) bits) is larger than the Delete-and-Conquer scheme. In fact. Fig. 5(b) 
experimentally shows that total amount of feedback sent by the Delete-and-Conquer decoder is strictly 
less than k bits. 

Remark 3 (broadcast scenario): To generalize the Delete-and-Conquer codes to the broadcast scenarios, 
we note that excluding a subset of recovered symbols from subsequent transmissions may increase the 
total number of transmissions (compared with when all recovered symbols are excluded), but it does not 
impede the decoding progress. In the worst case, no symbol is dropped from the encoding set, which 
reduces our codes to the original LT codes. Therefore, in a broadcast scenario, the encoder can simply 
take the intersection of collected feedbacks from different receivers, and proceed with excluding those 
symbols confirmed to be recovered by all receivers. 

V. Short Block Length Analysis 

In this section, we precisely analyze the performance gains of the primitive form of our codes for very 
short block length of k = 2 and k = 3 symbols. Although such small block lengths are not practical, they 
provide some insight into the Delete-and-Conquer scheme. For larger block lengths, our exact calculation 
of overhead in terms of degree probabilities becomes unwieldy. For the analysis purposes, we assume 
that the forward channel is lossless. 












A. Block length k—2 

As the first case, we consider the hlock length of /c = 2 symbols, in which two input symbols xi 
and X 2 are encoded. We assume that the probability of degree 1 transmission is equal to 2p, and the 
probability of degree 2 transmission is 1 — 2p. Therefore, an encoded symbol is equal to xi or X2 each 
with probability p, and xi+ X2 with probability 1 — 2p. 

Lemma 1. For the block length k = 2, if the probability of degree 1 transmission is 2p, then the 
Delete-and-Conquer codes require an expected number of forward transmissions and 2p feedback 
transmissions for successful decoding. 

Proof A Delete-and-Conquer decoder can successfully decode two symbols within n = 2 transmissions 
under the following possibilities for the received symbols: 

{xi,X2},{x2,Xi},{xi +X2,Xi},{xi +X2,X2}. 

The probability of terminating after two transmissions is obtained as 4p — 4p^. Similarly, the decoder 
would successfully recover xi and X 2 within n > 3 transmissions in the case of the following received 
symbols: 

n—1 symbols n—1 symbols 

{xi + X2, -I- X2,Xi}, {xi + X2, -f X2,X2}. 

The probability of successful recovery in this case would be: 

Q{n) = {I - 2p)'^-^ {p + p), n>3; 

and therefore, the expected number of forward transmissions for the Delete-and-Conquer scheme is equal 
to: 

^ 4j,2 , 1 

noei = 2(4p - 4p^) -h ^ nQ{n) = —--. (5) 

n=3 ^ 

To calculate the expected number of feedbacks transmitted, we note that one feedback is transmitted 
only in the cases of received symbols {xi,X 2 } and {x 2 ,xi} each happens with probability p, and thus 
the expected number of feedbacks transmitted would be 2p. It should be noted that the last feedback 
message is excluded from the count, as it is also needed by other coding schemes to stop the encoder 
from further transmissions. □ 

Theorem 1. For the block length k = 2, the Delete-and-Conquer codes provide a savings of in 
forward transmissions compared with the LT codes. 

Proof. First, we calculate the expected number of transmissions required by the LT codes to recover all 
symbols. To this end, we obtain the probability of full recovery within n>2 transmissions. For instance. 




Fig. 6. State space of the Delete-and-Conquer scheme with 3 input symbols. The four states inside the box are considered as 
a single state. Notation XiXj represents the symbol Xi + Xj, and dotted red lines represent transitions with a feedback. 


in the case of n = 2, the decoder should receive one of the following comhinations to successfully 
recover xi and X 2 '- 


{XI,X2},{XI,XI +X2},{X2,X1},{X2,X1 + X 2 }, {xi +X2,Xl},{xi +X2,X2}. 

Accordingly, the prohahility of decoding within two transmissions can he calculated as Ap — 6p^. For a 
general case of n transmissions, one can see that the prohahility of recovery within n transmissions is: 

P{n) = (p + (1 - 2p)) + (1 - 2p)"'~^ (p + p); 


and hence, the expected total number of transmissions is: 

00 A 9 ,1 

\ 4p — p + I 

niT = > nP[n) = ——--. 

^2 2p(l-p) 

Using (5) and (6), expected amount of savings ult — noei is obtained. 


( 6 ) 

□ 


B. Block length k—3 

For the block length k = 3, the authors in [13] have derived the expected number of encoding symbols 
required by the LT codes for full recovery. In this model, the set of received symbols at the decoder 
defines a state of an absorbing Markov chain, and the process (i.e., transmission of encoded symbols) 
ends when it reaches to the absorbing state that includes all input symbols decoded. We similarly adapt 
this approach to obtain the Markov chain for the Delete-and-Conquer scheme with 3 input symbols. The 




















































corresponding Markov chain shown in Fig. 6 , includes states up to the permutations of input symbols, 
e.g., two states {xi,X 2 +X 3 } and {x 2 ,xi +X 3 } are isomorphic and it is enough to consider a single unique 
state for each group of isomorphic states. In this figure, darker states are irreducihle hy the decoder in 
that no symbol can be further recovered, whereas other states can be immediately reduced by the decoder 
to the darker ones.. By constructing the state transition matrix P as 


P = 



we can compute the expected number of steps (transmissions) from the initial state to the absorbing state 
{xi,X 2 ,X 3 }. In the notation, matrix Q represents the transition probabilities between transient states, R 
denotes the probabilities between transient states and the absorbing state, and I is an identity matrix. 


Theorem 2. For the block length k = 3, given that pj is the probability of transmitting an encoded 
symbol with degree j, the expected number of transmissions required by the Delete-and-Conquer scheme 
for successful decoding is: 

- ^ J_ P 2 P 2 _ _ 3pi - 4p2 + 3pip2 -3pl + 3 

Pi 3pi + 2p2 P 1 +P 2 {pi + 2p2){p2 - 3) 3-p2 

Proof In an absorbing Markov chain with a transition matrix P and the fundamental matrix 

N = I + Q + Q2 + ... = (I-Q)~\ 

the expected number of steps (transmissions) from the initial state to the absorbing one is: 


n = ttoNc, ( 8 ) 

where ttq = (1 0 ... 0 ) is the initial probability corresponding to the state of no symbols been transmitted, 
and c = (1 .. 1)^ [18]. From Fig. 6 , we obtain matrix P as: 
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where we assume that after each symbol deletion at the encoder, the probabilities are normalized by 
dividing by the sum of the remaining degrees. For instance, after one exclusion, p'^ = and P 2 = 



^ 2 — would be the probability of degrees 1 and 2 transmissions respectively. This leads to the theorem 

□ 


P1+P2 

statement. 


The expected number of transmissions for the LT codes has been derived in [13] as follows: 

= J_ + 6pi ___9pi_ 

Pi Pi - 3 (3-p2)(3 - 2pi-P 2 ) 2(pi+p2)(3pi + 2p2) 

If the encoder uses only degree 1 symbols (i.e., pi = 1), the expected number of required symbols 

for the LT codes is = 5.5, illustrating the effect of the coupon collector’s problem; on the other 

hand, Delete-and-Conquer scheme requires only noei = 3 encoded symbols, which is the minimum 

possible number of forward transmissions. It should be noted that in this case, Delete-and-Conquer 

scheme turns into a no-coding ARQ method. An optimization in [13] results in a minimum number of 

4.046 forward transmissions (with pi = 0.524, p 2 = 0.366, and p 3 = 0.109) for the LT codes, whereas 

Delete-and-Conquer coding with these same probabilities yields a total number of = 3.678 forward 

transmissions. In general, we can numerically compare (7) to (9) to see that Delete-and-Conquer scheme 

can decrease the total number of forward transmissions up to 2.4-fold. 


Theorem 3. For k = 3 input symbols, the expected number of feedbacks transmitted by the Delete-and 
Conquer scheme before conclusion (i.e. not including the termination signal) is: 

3pi 


foel — 




( 10 ) 


3pi + 2p2 3 - P2 Pi + P2 

Proof. In an absorbing Markov chain, the probability of ever visiting state j when starting at a transient 
state i is the entry hij of the matrix H = (N — I)N^g, where N is the fundamental matrix and Ndg is 
the diagonal matrix with the same diagonal as N, and I is an identity matrix [18]. In Fig. 6 , a feedback 
is transmitted when transitions along the dotted-line occur, e.g. a transition from the state 1 to state 2 . 
Accordingly, the probability of such transitions, and hence the expected number of feedbacks transmitted 
is given by: 


foel — hi2 + hi2h25 + /il3^37 + /ll3^38 + hi^h^s', 

from which the result follows. 


□ 


Based on the Theorem 2 and 3, we can calculate the optimal probability values and P 2 (and 
P 3 = 1 — — P 2 ) that minimize the total number of forward and feedback transmissions needed by the 

Delete-and-Conquer scheme. In other words: 


= argmin [noei + foei] ; 

(Pl,P2) 



which results in P 2 ) = (0.644, 0.206) (and pg = 0.150) with a minimum number of total transmissions 
4.7247. In this case, we simply considered the sum of forward and feedhack transmissions. In a more 
general sense, we can assume that each transmission through the forward channel has a cost of Ci, while 
each feedhack transmission has a cost of € 2 - Therefore, the optimal prohahility values can he calculated 
as: 

{Pi^pD = argmin [CifiDei + C2fDei] ■ 

(Pl>P2) 

Moreover, in comparison with the LT codes, one can notice that it is worthwhile to send feedhack if: 

ClflDel + C2fDel < CiUlT - < ^T, 

UlT — noel W2 

where ult^ and are calculated in (7), (9), and (10). 

VI. Maximum-Likelihood Decoder Analysis 

In this section, we derive an upper-hound on the failure prohahility of the maximum likelihood (ML) 
decoder when used with the Delete-and-Conquer codes. We assume that there are k input symbols at 
the transmitter, and that n encoding symbols are received over a binary erasure channel (BEC). The ML 
decoding over a BEC is equivalent to recovering k information (input) symbols from n received encoding 
(output) symbols. Without loss of generality, we assume that each symbol is one bit; x is a row vector 
containing k input bits; and y is the vector of n output bits. Matrix G = [gij] is an n x /c adjacency 
matrix of the decoder graph, such that an entry gij is equal to 1 if the output node has the input 
node as a neighbor. The ME decoder is then equivalent to solving a system of linear equations (with 
unknowns x and received symbols y) of the form: 

Gx^ = y^. (11) 

Encoding symbols with a distance of 0 or 1 trigger a feedback message that causes the corresponding 
symbols to be excluded from future transmissions. Excluding the recovered symbols from subsequent 
transmissions is equivalent to setting the subsequent elements of the corresponding columns in G to zero. 
Eor instance, Eig. 7 shows a realization of the matrix G in which the first feedback message acknowledges 
recovery of X 2 - Thereafter the second column of G (i.e., the shaded part) is set to zero. 

The ME decoder failure is equivalent to the event that the adjacency matrix G in (11) is not of full 
rank. Eet pg be the probability that an input bit j (for an arbitrary j G {1,2,..A:}) is not recoverable 
under the ME decoding rule. Erom [19]: 
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Fig. 7. Decoder matrix and the location of feedback transmission 
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Fig. 8. Location of feedbacks and interval of coding 


Pe = Pr{3x G GF(2^),a;j = 1 : Gx^ = 0^} < ^ Pr{Gx^ = 0^}. (12) 

xeGF(2'') 

Xj = l 

In order to calculate Pr{Gx^ = 0^}, we separately consider the rows of G between consecutive 
feedback messages. We assume that L feedback messages are transmitted in total such that after receiving 
ti encoding symbols the first feedback message is transmitted, after receiving t 2 encoding symbols the 
second feedback is sent, and so forth. At the boundary points, we define fo = 0 and tp = n. Therefore, 
there is no feedback within each interval of [0, ti], (ti, t 2 ], ..., (tp-i, n], and there is one feedback at 
the end of each interval, as shown in Fig. 8. We assume that within the interval (z = 0,..., L — 1) the 
coding window contains k — rrii symbols, and thus the encoder uses a fixed degree distribution flk-mi {d) 
defined over the set of k — rui unacknowledged symbols. 

To calculate an upper bound on the decoder failure probability, we start with a single row of G. Let 
r to be a row of degree d, and assume that the total number of m symbols are acknowledged before 
transmitting r; in other words, m indices out of k indices in r are forced to be zero. We define a row 
vector f such that /; = 1 if the symbol has been acknowledged, and 0 otherwise (i.e., an indicator 

































function on the index of acknowledged symbols). For a given input vector x with ||x||o = w (||.||o is 
the 0-norm), we have the following lemma: 


Lemma 2. Given that the row vector r has degree d (i.e., ||r||o = d), the probability o/rx^ = 0 is: 

( k—m—w\ 


FIX, ||r||o 


= d) = 


E (w\ (k—m—w\ 

M=0,2,..,min(2[|J,rD) Vtt/V d—u ) 

fk—m\ 
d ) 


in which re = re — (x, f) with (x, f) denoting the dot product of two vectors. 

Proof. The event rx^ = 0 happens if and only if r has an even number of I’s in those indices of j in 
which Xj is equal to 1 as well. Assume that J = {ji,j 2 , is the set of indices in which x is 1, and 

A = {ai,a 2 , is the set of acknowledged indices. Therefore, we need to choose an even number 

u of indices that belong to J but not to A. The number of these non-overlapping indices is given by 
re = re — (x, f). Because the degree of r is d, we then need to choose d — u symbols from the remaining 
/c — m — re -|- (x, f) indices that belong neither to J nor to A. Finally, given that the vector r is generated 
randomly (i.e., d neighbors are selected uniformly at random from k — m unacknowledged symbols), the 
result follows. □ 

Using Lemma 2 and the fact that r has degree d with probability Qk-m{d), we have: 

k—m 

p(x) = ^ ||r||o = d). (13) 

d=l 

Now, we can extend this result to more than one row of G in the following manner. Let us denote the 
ti — fj+i rows of G by G*. Rows in G* are generated independently according to the degree distribution 
Qk-m,i{d). Thus, we have: 

Pr{G,x'^ = 0^} = (pi(x))*^+^-*^ , (14) 

in which pj(x) is calculated as in (13), and based on the number of acknowledged symbols and the 
degree distribution within the interval, i.e.,: 

k—mi 

Pi{^) = Ufc_m.(d)pj(x, ||r||o = d). 
d=l 

Given that there are L transmit intervals (i.e., L feedback messages), we can calculate the probability of 
Gx^ = 0^ for a given vector x as follows: 

Pr{Gx'^ = O'^} = PrjGix'^ = o'^}. 

i=0 


( 15 ) 



Assembling these steps together, the ML decoder failure probability of the Delete-and-Conquer scheme 
is given by the following theorem. 


Theorem 4. Given that L feedbacks are transmitted in total (i.e, one feedback after receiving the ti-th 
(i = encoding symbol), the ML decoder failure probability of recovering an input symbol j (for 

an arbitrary j G {1, 2, ..k}) is upper bounded by 

f k L—1 


Pe < min 


rE( E 


W = 1 


X 


||x||o=-!« 

Xj = l 


PrjGix'^ 

i=0 



(16) 


Proof From Lemma 2 and its following results, we obtain that for a given input vector x with Hamming 
weight w and L feedback messages, the probability of Gx^ = 0^ is calculated as in Eq. (15). Therefore, 
summing over all possible input vectors x with the index equal to 1, yields the theorem statement. 
It should be noted that we assume the values of L and fj’s (i.e., the total number of feedbacks and their 
trigger points) are known. □ 

From [19], the upper bound on the ML decoder failure probability when there is no feedback is 
calculated as: 



Fig. 9 numerically compares the upper bound in (16) with that in (17) for k = 100 input symbols. 
The results confirm that collecting more encoding symbols reduces the bound on MF failure probability, 
as expected. However, Delete-and-Conquer codes with 0 and 1 feedback messages achieve a tighter 
upper-bound on the decoder failure probability. 

Asymptotic results: We conclude our analysis of the Delete-and-Conquer scheme by providing upper 
bounds on its performance metrics. To this end, we keep the assumption that initially there exist k input 
symbols at the encoder, and at some point, m symbols are acknowledged. The Delete-and-Conquer 
distribution is thus given by Llk-m{i) (for i = 1,...,A: — m). Adapting the results of [3] yields that 
the average degree of an encoding symbol generated by the encoder is given hy D = 0(ln(A: — m)). 
Furthermore, an encoder that deletes m symbols out of k symbols, needs to transmit at most k — 
m + O {s/k — m ln^(^^)) encodings so that the decoder be able to recover all input symbols with 
probability at least 1 — <5. Furthermore, computational complexity of the coding process is given by 
0{{k — m)ln^-j^). One can notice that as the number of acknowledged symbols (i.e., parameter m) 
increases, performance metrics improves. In the case of no feedbacks (i.e., m = 0) Delete-and-conquer 
codes reduce to the original FT codes. 




Fig. 9. Upper bound on the maximum likelihood decoder failure prohahility for the LT and the Delete-and-Conquer codes 


VII. Simulation Results 

We evaluate the performance of rateless codes with nonuniform selection distributions against the 
Growth codes, Online codes proposed in [16], and recently proposed LT-AF codes [17]. 

A. General form 

1) Intermediate performance: In many applications such as video streaming with real-time playback 
requirements, it is essential to partially recover some symbols before the recovery of entire frame. In this 
context, although LT codes are capacity-achieving, they lack real-time features; in other words, not many 
input symbols are decoded until the decoding process is almost complete. By incorporating a nonuniform 
selection distribution at the encoder, we aim to enhance the intermediate symbol recovery rate. Fig. 10 
compares the performance of our codes with the LT-AF codes of Variable Node with Maximum Degree 
(LT-AF-i-VMD) [17], where the authors show that LT-AF codes can surpass previous rateless codes with 
feedback including SLT codes. One key point, however, is that the LT-AF decoder is not able to recover 
any symbol until at least k encoding symbols are received. As the results show, our scheme based on 
the Quantized distance method can achieve a high intermediate recovery rate. Moreover, the coding 
performance can be adjusted by tuning the parameter s (feedback transmission interval). 

2 ) Coding overhead: Next we compare the total number of forward and feedback transmissions needed 
by our codes in comparison with the LT-AF-fVMD codes. As the results in Table I show, our codes have a 
slightly better performance in terms of number of forward transmissions. However, LT-AF codes require 










Fig. 10. Intermediate performance of codes with nonuniform symbol selection against the LT-AF codes (k = 512). 
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less feedback transmissions. It should be noted that amount of feedback in our codes can be adjusted 
using the parameter s, and that our codes are aimed to achieve a high intermediate symbol recovery 
rate, as the results in Fig. 10 show. Table II shows the performance of our codes with the block length 
k = 512 symbols and as the feedback interval s increases. Similar to the previous results, the encoder is 
able to control the number of forward and feedback transmissions by changing the parameter s. 

B. Primitive form 

1) Intermediate performance: To investigate the progressive performance of the Delete-and-Conquer 
codes, we run simulations with the block length of A: = 512. Results shown in Fig. 11, demonstrate 
that Growth codes can provide higher symbol recovery rate at the beginning, while Delete-and-Conquer 
achieves better performance when a small fraction of symbols are unrecovered (near the “knee”)- On the 
other hand, Delete-and-Conquer scheme achieves better performance compared with the Online codes. 
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Fig. 11. Intermediate performance of the Delete-and-Conquer codes compared with other rateless codes (fc = 512) 


noting that Delete-and-Conquer codes improve the intermediate performance with a lightweight utilization 
of the hack channel (i.e., one hit feedback for each of a small fraction of received symbols). 

2) Computational complexity: Computational costs at the encoder and decoder are mainly related to 
the average degree of input symbols. Fig. 12 shows the average degree of input symbols for different 
codes compared to the Delete-and-Conquer codes. As the results show, Delete-and-Conquer codes have 
a smaller average degree on input symbols, and hence they incur less computational complexity. Smaller 
average degree is due to incrementally dropping input symbols from the coding window. 




















Fig. 12. Average degree of input symbols for various coding schemes 


VIII. Conclusion 

In this paper, we have developed feedhack-hased rateless codes with a nonuniform selection distribution. 
Our encoders estimate the decoder state using feedback information, and dynamically adjust the selection 
distribution so that more helpful symbols (in terms of decoding progress) are assigned with a higher 
probability to be included in future encodings. As a result, we improve the intermediate performance 
of the underlying rateless codes and make them more suitable for applications with real-time decoding 
requirements. Our codes further support two important features: our decoder has full control of the rate 
and timing of feedback transmission. Our simulation results, backed by analysis, confirm that distance- 
type feedback paired with a nonuniform selection distribution achieves a high intermediate recovery rate. 
On the whole, rateless codes with nonuniform selection distributions help the encoder to optimize for 
the performance requirements dictated by the application. 
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